In-situ display monitoring and calibration system and methods

ABSTRACT

Disclosed are embodiments of in-situ display monitoring and calibration systems and methods. An image acquisition system captures images of the viewing plane of the display. Captured images may then be processed to characterize various visual performance characteristics of the display. When not in use capturing images of the display, the image acquisition system can be stored in a manner that protects it from environmental hazards such as dust, dirt, precipitation, direct sunlight, etc. A calibration image in which a plurality of light emitting elements is set to a particular color and intensity may be displayed, an image then captured, and then a difference between what was expected and what was captured may be developed for each light emitting element. Differences between captured images and expected images may be used to create a calibration data set which then may be used to adjust the display of further images upon the display.

CROSS REFERENCE TO RELATED APPLICATIONS

This non-provisional utility application claims the benefit of and is acontinuation of application Ser. No. 16/245,792, filed Jan. 11, 2019 andentitled “In-Situ Display Monitoring and Calibration System andMethods”. Application Ser. No. 16/245,792 claimed the benefit of and isa continuation of application Ser. No. 15/459,089, filed Mar. 15, 2017and entitled “In-Situ Display Monitoring and Calibration System andMethods”. Application Ser. No. 15/459,089 claimed the benefit of priorfiled provisional application No. 62/309,739, filed Mar. 17, 2016 andentitled “In-Situ Display Monitoring and Calibration System andMethods”. Application Ser. No. 15/459,089 and 62/309,739 are hereinincorporated by reference.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

Not Applicable.

DESCRIPTION OF ATTACHED APPENDIX

Not Applicable.

BACKGROUND

The sense of sight is utterly compelling to those human beings whopossess it. The adage that a picture is worth a thousand words resonateswith an appreciation of the profound importance of taking in visualinformation. The sense of sight is unique in allowing us to absorb somuch information from our world so quickly. It is natural then thatadvertisers, entertainers, artists, and others all want to engage peoplewith their own visual content for the purpose creating a desiredresponse in their intended audience. A large scale visual display systemis a particularly compelling way for people to experience thepresentation of visual information and such systems are the focus of thepresent disclosure.

There are numerous features of a visual display system that contributeto its impact upon viewers including: size, brightness, contrast, colorsaturation, color depth, display refresh rate, resolution, pixel pitch,pixel pitch uniformity, and others.

There are numerous other features of a visual display system that are ofinterest to the owners and operators of such systems including: ease ofinstallation, ease of service, reliability, ease of configuration, easeof maintenance, ease of operation, cost of the system, cost ofinstallation, cost of operation, cost of service, and others.

Display systems with large screen sizes present a number of difficultproblems that are in need of solution. One significant challenge fordisplay owners and operators is to maintain the visual performance of alarge display once it has been installed in a viewing location. Thevisual performance of a display can be characterized using a number ofmeasures including: uniformity of brightness across the entire display,uniformity of color across the entire display, contrast ratio, colortemperature and uniformity of color temperature across the entiredisplay, color fidelity to predetermined standards, etc.

Large displays may be built from a plurality of individual lightemitting elements, arranged in a pre-determined pattern to create acomposite viewing plane. Due to variances in manufactured materials andmanufacturing processes, it is a fact of life that characteristics ofindividual light emitting devices vary from one device to the next andthat individual devices may respond differently to environmentalconditions of voltage, current, temperature, humidity, exposure to sunlight, exposure to atmospheric gases such as ozone and nitrogen oxides,and aging. Variations in performance of individual light emittingelements include the characteristics of: luminous intensity per lightemitting element, luminous intensity produced per unit current, dominantwave length of emitted light, wavelength distribution of emitted light,temperature coefficient of change of any of the prior parameters. Visualperformance of the entire display is therefor subject to change as eachand every light emitting element is exposed to the previously listedenvironmental factors and their variations over time.

In consideration of the foregoing points, it is clear that embodimentsof the present disclosure confer numerous advantages and are thereforehighly desirable.

SUMMARY

The present disclosure is directed to systems and methods for monitoringand calibrating display screens which comprise a plurality of displaymodules, each module having a plurality of light emitting elements, theplurality of display modules disposed to collectively provide a viewingplane. Other aspects of the disclosure are directed to systems andmethods for monitoring and calibrating display screens which comprise aplurality of display modules, each module having a plurality ofmodulated reflective elements disposed to collectively provide a viewingplane.

Display systems of the present disclosure comprise a plurality of lightemitting elements coupled to a substrate and arranged in a predeterminedpattern collectively forming a viewing plane. Other display systems ofthe present disclosure may comprise a plurality of display modulesassembled to make a large, unified, visual display in which each displaymodule comprises a plurality of light emitting elements coupled to asubstrate and arranged in a predetermined pattern with respect to aviewing plane. Each display module may be shaped so that it may abut oneor more other display modules without introducing gaps or overlapsbetween adjacent display modules. The display systems disclosed create ahighly uniform visual effect by creating highly uniform spacing betweenlight emitting elements, both within a single display module and acrossa plurality of display modules when the plurality are assembled into alarge, unified, visual display.

The present disclosure provides systems and methods of monitoring andcalibration of displays that may be in indoor or outdoor locations.Basic system features include: an electro-optical image acquisitionsystem; a moveable linkage coupled to the image acquisition system, thelinkage having at least two defined positions: a deployed position inwhich the image acquisition system is disposed and directed so that theviewing plane of the display is imagable by the image acquisitionsystem; and a stored position in which the image acquisition system isprotected from the environment. The image acquisition system has spatialresolution and sensitivity to both color and luminous flux sufficient toenable monitoring and calibration operations using data acquired by theimage acquisition system.

Each time the moveable linkage is moved into the deployed position, theimage acquisition system is disposed in the same position andorientation with respect to the position and orientation of the viewingplane of the display. Each time the moveable linkage is moved into thestored position, the image acquisition system is disposed to protect itfrom the environment, direct sun exposure, precipitation, etc.

Raw data acquired by the image acquisition system may be processed toprovide normalized image data from the display. In turn, normalizedimage data may be processed to provide calibration data for use incalibrating the display's visual performance according to desiredcharacteristics. In turn, calibration data may be processed by thedisplay to affect, adjust, or perfect the visual performance of thedisplay.

Acquisition of raw data by the image acquisition system may occurautonomously or under the control of a remote agent. Autonomousacquisition may occur according to a pre-established calendar orschedule. Autonomous acquisition may furthermore be condition responsivewith respect to ambient lighting, temperature, time-of-day, or weatherconditions. For example, it may be advantageous to delay acquisition ofraw data if it is raining or snowing outside. Condition responsiveacquisition may delay or otherwise schedule acquisition until localconditions are more suitable.

Raw data may be acquired from the image acquisition system and processedto form calibration data, which can then be acted upon locally toaccomplish an in-situ adjustment to the visual performance of thedisplay. In addition, raw data acquired by the image acquisition systemmay be transmitted from the display to a remote entity, therebyfacilitating processing of the raw data by a remote entity. Calibrationdata may be computed remotely and then transmitted to the displaysystem, which can then act locally to accomplish an in-situ adjustmentto the visual performance of the display.

When in the deployed state, the image acquisition system is disposedwith respect to the viewing plane of the display in a definite positionand orientation. This establishes a known geometric relationship betweenthe viewing plane and the image acquisition system. Under certaincircumstances the previously mentioned geometric relationship may causeundesirable non-uniformities in the raw data. Many of the feasiblegeometric relationships between viewing plane and image acquisitionsystem result in raw data that captures more light from some regions ofthe viewing plane and less light from other portions of the displayplane.

The known geometric relationship may be combined with the knownelectro-optical properties of the image acquisition system to form anormalization function that substantially counteracts the undesirablenon-uniformities arising from the known geometric relationship. The stepof normalizing the acquired raw data may precede the step of formingcalibration data. Use of the calibration data may then proceed asbefore.

A sequence of one or more calibration patterns may be displayed on theviewing plane. Raw data may be captured by the image acquisition systemcorresponding to the one or more calibration patterns. The captured datamay be used singly or jointly by means of one or more statisticalcombinations of more than one image. Raw data may then be processed toproduce calibration data for the display. The calibration data maypertain to a plurality of individual light emitting elements of thedisplay or may apply to regions containing a plurality of light emittingelements. Calibration data may comprise data corresponding to one ormore of the following visual performance characteristics of the display:white point, color gamut, color balance, gamma correction, andbrightness.

Exemplary Embodiment 1.0

According to an embodiment of the present disclosure, a monitoringsystem for use with a display having a plurality of light emittingelements arranged in a predetermined pattern collectively forming aviewing plane comprises:

an image acquisition system triggerable to capture one or more images,each captured image comprising an image of at least a portion of theviewing plane, said image acquisition system comprising a plurality oflight receiving elements arranged in a predetermined patterncollectively forming an imaging plane;

an actuatable linkage coupled to said image acquisition system, saidlinkage operative to move said image acquisition system between adeployed position and a stored position;

said deployed position characterized in that said image acquisitionsystem is disposed in a predetermined position to capture an image of atleast a portion of the viewing plane;

said stored position characterized in that said image acquisition systemis substantially protected from airborne particulates and issubstantially protected from both condensed and non-condensedatmospheric moisture.

Exemplary Embodiment 1.1

According to another embodiment of the present disclosure, exemplaryembodiment 1.0 is further characterized in that: the image acquisitionsystem is triggerable to capture one or more images, each captured imagebeing an image comprising the entire viewing plane; and, the deployedposition is further characterized in that said image acquisition systemis disposed in a predetermined position to capture one or more images ofthe entire viewing plane.

Exemplary Embodiment 1.2

According to another embodiment of the present disclosure, exemplaryembodiment 1.1 is further characterized in that each light emittingelement of said viewing plane is uniquely imagable by at least one ofsaid plurality of light receiving elements forming said imaging plane.

Exemplary Embodiment 1.3

According to another embodiment of the present disclosure, exemplaryembodiment 1.1 is further characterized in that the display comprises aplurality of display modules, each display module comprising a pluralityof light emitting elements arranged in a predetermined pattern, saidplurality of display modules collectively forming the viewing plane; thesystem further characterized in that each display module of said viewingplane is uniquely imagable by at least one of said plurality of lightreceiving elements forming said imaging plane.

Exemplary Embodiment 2.0

According to another embodiment of the present disclosure, any ofexemplary embodiments 1.0, 1.1, or 1.2 may be further characterized inthat: a surface normal vector is defined at each of said plurality oflight emitting elements perpendicular to said viewing plane; anincidence vector is defined for each of said plurality of light emittingelements starting at each of said plurality of light emitting elementsand directed toward the portion of the imaging plane that images each ofsaid light emitting elements, each incidence vector having both adirection and a distance; each light emitting element of the displayproducing a first luminous output in the direction of said surfacenormal and a second luminous output in the direction of the incidencevector, said first and second luminous outputs being in a substantiallyknown relationship; the system additionally comprising a data processingmeans operable to compute a normalized calibration image by applyingsaid substantially known relationship to one or more images captured bysaid image acquisition system, the normalized calibration imagecomprising an estimate of said first luminous output for each of saidplurality of light emitting elements.

Exemplary Embodiment 2.1

According to another embodiment of the present disclosure, exemplaryembodiments 1.0, 1.1 or 1.2 may be further characterized in that: aviewpoint vector is defined at each of said plurality of light emittingelements, each viewpoint vector being directed in the same directionfrom each of said plurality of light emitting elements; an incidencevector is defined for each of said plurality of light receiving elementsstarting at each of said plurality of light emitting elements anddirected toward the portion of the imaging plane that images each ofsaid light emitting elements, each incidence vector having both adirection and a distance; each light emitting element of the displayproducing a first luminous output in the direction of said viewpointvector and a second luminous output in the direction of the incidencevector, said first and second luminous outputs being in a substantiallyknown relationship; the system additionally comprising a data processingmeans operable to compute a normalized calibration image by applyingsaid substantially known relationship to one or more images captured bysaid image acquisition system, the normalized calibration imagecomprising an estimate of said first luminous output for each of saidplurality of light emitting elements.

Exemplary Embodiment 2.2

According to another embodiment of the present disclosure, any ofexemplary embodiment 1.3 may be further characterized in that: a surfacenormal vector is defined at each of said plurality of display modulesperpendicular to said viewing plane; an incidence vector is defined foreach of said plurality of display modules starting at about the centroidof the display plane of each of said plurality of display modules anddirected toward the portion of the imaging plane that images each ofsaid display modules, each incidence vector having both a direction anda distance; each display module of the display producing a firstluminous output in the direction of said surface normal and a secondluminous output in the direction of the incidence vector, said first andsecond luminous outputs being in a substantially known relationship; thesystem additionally comprising a data processing means operable tocompute a normalized calibration image by applying said substantiallyknown relationship to one or more images captured by said imageacquisition system, the normalized calibration image comprising anestimate of said first luminous output for each of said plurality ofdisplay modules.

Exemplary Embodiment 2.3

According to another embodiment of the present disclosure, exemplaryembodiment 1.3 may be further characterized in that: a viewpoint vectoris defined at each of said plurality of display modules, each viewpointvector being directed in the same direction from each of said pluralityof display modules; an incidence vector is defined for each of saidplurality of display modules starting at about the centroid of thedisplay plane of each of said plurality of display modules and directedtoward the portion of the imaging plane that images each of said displaymodules, each incidence vector having both a direction and a distance;each display module of the display producing a first luminous output inthe direction of said viewpoint vector and a second luminous output inthe direction of the incidence vector, said first and second luminousoutputs being in a substantially known relationship; the systemadditionally comprising a processing means operable to compute anormalized calibration image by applying said substantially knownrelationship to one or more images captured by said image acquisitionsystem, the normalized calibration image comprising an estimate of saidfirst luminous output for each of said plurality of display modules.

Exemplary Embodiment 2.4

According to another embodiment of the present disclosure, exemplaryembodiment 2.0, 2.1, 2.2, or 2.3 further characterized in that first andsecond luminous outputs comprise one or more of the followingproperties: luminous intensity, wavelength of luminous output.

Exemplary Embodiment 3.0

According to another embodiment of the present disclosure, exemplaryembodiments 2.0, 2.1, 2.2, 2.3, or 2.4 further comprising: a displaycontrol system operable to render visual data on said display, saiddisplay control system responsive to one or more of said normalizedcalibration images to change the rendering of visual data upon at leasta portion of the display.

Exemplary Embodiment 3.1

According to another embodiment of the present disclosure, exemplaryembodiment 3.0 in which the change in rendering of visual data upon thedisplay changes one or more of the following visual characteristics ofat least a portion of the display: white point, color gamut, colorbalance, gamma correction, gray-scale rendering and brightness.

Exemplary Embodiment 3.2

According to another embodiment of the present disclosure, exemplaryembodiment 3.0 or 3.1, the display control system further characterizedin that the rendering of visual data on the display comprises the stepsof:

receiving visual media data, the visual media data comprising brightnessand color information for each of a plurality of picture elements at anencoded resolution;

transforming the received visual media data by combination with displayspecific calibration data thereby forming a set of corresponding visualdata at a display resolution, the calibration data comprisingadjustments to one or more of the following visual characteristics:white point, color gamut, color balance, gamma correction, andbrightness; and,

displaying the visual data on at least a portion of said viewing plane.

Exemplary Embodiment 4.0

According to another embodiment of the present disclosure, an in-situmonitoring and calibration system for a display, the display comprisinga plurality of light emitting elements collectively creating a viewingplane with a displayed resolution, the system comprising:

a display control system operative to:

-   -   receive visual media data comprising brightness and color        information for each of a plurality of picture elements at an        encoded resolution;    -   receive calibration data comprising adjustments to brightness        and color for each of a plurality of light emitting elements        comprising said display;    -   transform said visual media data using said calibration data        thereby forming a set of visual data at said display resolution,        said set of visual data corresponding to said visual media data;    -   display said set of visual data upon said the viewing plane of        said display;    -   display a sequence of one or more calibration images upon said        display;

an image acquisition system triggerable to capture one or more images,each captured image comprising an image of at least a portion of theviewing plane, said image acquisition system comprising a plurality oflight receiving elements arranged in a predetermined patterncollectively forming an imaging plane, further characterized in thatsaid image acquisition system is disposed in a predetermined position tocapture one or more images of the entire viewing plane;

said display control system further operative to trigger said imageacquisition system to capture one or more images of said one or morecalibration images and, for each calibration image displayed on saidviewing plane, create a normalized image.

Exemplary Embodiment 5.0

According to another embodiment of the present disclosure, a method forin-situ monitoring and calibration of a display, the display comprisinga plurality of light emitting elements collectively creating a viewingplane with a displayed resolution, the method comprising the steps of:

presenting a calibration image on the viewing plane of a display;

acquiring, with an image acquisition system having a plurality ofimaging elements collectively creating an imaging plane, a capturedimage of the displayed calibration image, each light emitting element ofthe viewing plane being in a known relationship to the region of theimaging plane that images each light emitting element;

applying said known relationship to said raw image to form a normalizedimage;

comparing said normalized image to said calibration image and generatinga calibration data set that encodes the differences between saidcalibration image and said normalized image;

applying said calibration data set, in a display control system, tomodify the rendering of one or more displayed images.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other features, aspects, and advantages of the presentinvention will become better understood with regard to the followingdescription, appended claims, and accompanying drawings where:

FIG. 1A shows a square consistent with a regular four sided polygon.

FIG. 1B shows a square tiling of a two dimensional plane

FIG. 1C shows coordinate axis defined on square tiling. Enlarged view 1Dis indicated

FIG. 1D shows an enlarged view of the indicated region of FIG. 1Cshowing uniform row and column pitch distance.

FIG. 1E shows a plan view of a display module having a plurality oflight emitting elements coordinate axis defined on square tiling.

FIG. 1F shows the display module of FIG. 1E overlaid with thepredetermined pattern of square tiling and coordinate axes.

FIG. 1G shows a plan view of two display modules aligned along theiry-axis.

FIG. 1H shows a plan view of two display modules aligned along theirx-axis.

FIG. 2A shows a perspective view of an in-situ display monitoring andcalibration system with a typical frame of visual media displayed on theviewing plane of the display. An image acquisition system is shown in astored position.

FIG. 2B shows a perspective view of the system of FIG. 2A with acalibration pattern displayed on the viewing plane of the display. Animage acquisition system is shown in a deployed position.Cross-sectional view 3A is indicated.

FIG. 3A is a cross-sectional view of an in-situ display monitoring andcalibration system that was indicated in FIG. 2B. Enlarged view 3B isindicated.

FIG. 3B shows an enlarged view, as noted in FIG. 3A, of an imageacquisition system.

FIG. 4A shows a light emitting element with axes X-X and Y-Y shown tofacilitate the understanding of FIG. 4B and FIG. 4C.

FIG. 4B shows a representation of the 1/r{circumflex over ( )}2dependence of illuminance with respect to distance from the lightemitting element of FIG. 4A.

FIG. 4C shows a representative graph of relative illuminance for thelight emitting element of FIG. 4A. FIG. 4C shows relative illuminance asa function of angle with respect to the X-X axis, and, relativeilluminance as a function of angle with respect to Y-Y axis.

FIG. 5A is a representation of a captured image, the captured imagebeing of a calibration pattern displayed on the viewing plane of adisplay. Visible in the figure are image features F1, F2, F3, and F4which correspond to known geometric features of the display.

FIG. 5B is a representation of a normalized calibration imagecorresponding to the calibration pattern captured in FIG. 5A.

FIG. 6 is a functional block diagram of an in-situ display monitoringand calibration system.

FIG. 7 is a schematic flowchart of a process for rendering visual mediadata on a display.

FIG. 8 is a schematic flowchart of a process for creating a normalizedimage.

FIG. 9 is a schematic flowchart of a process for calibrating a display.

FIG. 10A shows a perspective view of an in-situ display monitoring andcalibration system with a calibration image displayed on the viewingplane of the display. The system is shown comprising two imageacquisition systems which collectively image the entire viewing plane.

FIG. 10B is a representation of a captured image as captured by a firstimage acquisition system, the captured image being of a portion of acalibration pattern displayed on the viewing plane of a display. Visiblein the figure are image features F1 and F4 which correspond to knowngeometric G1 and G4 features, respectively, of the display.

FIG. 10C is a representation of a captured image as captured by a secondimage acquisition system, the captured image being of a portion of acalibration pattern displayed on the viewing plane of a display. Visiblein the figure are image features F2 and F3 which correspond to knowngeometric G2 and G3 features, respectively, of the display.

LIST OF REFERENCE NUMBERS APPEARING IN THE FIGURES

-   -   2—in-situ display monitoring and calibration system    -   4—display    -   6—display control system    -   8—coordinate system showing x-axis, y-axis, and z-axis    -   8X—x-axis    -   8X a, 8Xb—first x-axis, second x-axis    -   8Y—y-axis    -   8Ya, 8Yb—first y-axis, second y-axis    -   8Z—z-axis    -   10—square tile, which is a regular 4 sided polygon    -   10 a, 10 b, etc.—first square, second square, etc.    -   11—pitch distance    -   12—square tiling of the plane    -   12 v—representative vertex of the square tiling    -   12 s—representative side of the square tiling    -   14—predetermined pattern corresponding to a tiling of the plane    -   16—rectangular tiling of the plane    -   20—actuatable linkage    -   21—Communication network    -   22—data processing means    -   23—visual media data    -   24—calibration pattern    -   25—expected image    -   26—calibration data set    -   27—transformed visual media data    -   28—visual media rendered on the viewing plane of the display    -   29—configuration data    -   30—image acquisition system    -   30 a, 30 b—first, second image acquisition system    -   31—image acquisition system stored position    -   33—image acquisition system deployed position    -   34—plurality of light receiving elements    -   35—relative illuminance    -   36—imaging plane    -   38—captured image    -   40—normalized image    -   41—normalization function    -   50—incidence vector    -   50 a, 50 b, 50 c, . . . —first, second, third, etc. incidence        vector    -   70—display module    -   70 a, 70 b, 70 c, . . . —first, second, third, etc. display        module    -   71—light emitting element    -   71 a, 71 b, etc.—first, second, etc. light emitting element    -   72—plurality of light emitting elements    -   72 a, 72 b, etc.—first light emitting element, second light        emitting element, etc.    -   74—display plane    -   74 a, 74 b—first display plane, second display plane    -   75—display plane disposed at a first angle with respect to the        viewing plane    -   76—display module substrate    -   78—display assembly    -   78 a, 78 b, 78 c, etc.—first, second, third, etc. display        assembly    -   80—viewing plane    -   82—surface normal vector    -   84—luminous output    -   84 a, 84 b, . . . —first, second, etc. luminous output    -   86—viewpoint vector    -   86 a, 86 b, . . . —first, second, etc. viewpoint vector    -   {i, j, k}—unit vectors in x, y, and z directions, respectively    -   {x₀, y₀, z₀}—location of the center of the imaging plane in 3        dimensions    -   {x_(i), y_(i), z_(i)}—location of the i^(th) light emitting        element in 3 dimensions    -   {a₁, b₁, c_(i)}—direction cosines corresponding to the i^(th)        incidence vector, the incidence vector having the form:        a_(i)*i+b_(i)*j+c_(i)*k    -   G₁, G₂, G₃, G₄—first, second, third, fourth geometric feature of        the viewing plane    -   F₁, F₂, F₃, F₄—first, second, third, fourth geometric feature        identified in a captured image, in which F₁ corresponds to G₁,        F₂ corresponds to G₂, F₃ corresponds to G₃, and F₄ corresponds        to G₄.    -   r0, r1, r2—first, second, third radial distances from a        reference point    -   A0—area located at a distance of r0    -   A1—projection of area A0 at distance r1    -   A2—projection of area A0 at distance r2    -   200—a process for rendering visual media on a viewing plane    -   202—process step of receiving, in a display control system, a        frame of visual media data    -   204—process step of transforming a frame of visual media data in        a display control system    -   206—process step of displaying a transformed frame of visual        media    -   300—a process for creating a normalized image    -   302—process step of positioning in a deployed position with        respect to a display, an image acquisition system    -   304—process step of triggering an image acquisition system to        acquire a captured image    -   306—process step of defining an incidence vector for each of a        plurality of light receiving elements    -   308—process step of associating with each of the light receiving        elements comprising the imaging plane a normalization function    -   310—process step of applying a normalization function to a        captured image thereby producing a normalized image    -   400—a process for calibrating a display    -   402—process step of displaying a calibration pattern    -   404—process step of associating an expected image with a        calibration pattern    -   406—process step of triggering an image acquisition system to        acquire a captured image of a viewing plane    -   408—process step of creating a normalized image from a captured        image    -   410—process step of forming a calibration data set comprising        the color and brightness differences between an expected image        and a normalized image    -   412—process step of applying, in a display control system, a        calibration data set to the rendering of visual media upon a        viewing plane of a display such that the differences between a        normalized image and an expected image are reduced

DESCRIPTION

Uniformity in color, brightness, gray-scale are fundamental visualperformance goals for a large display. Any visual non-uniformity presenton the viewing plane of the display is easily noticed by viewers due tothe highly refined and discriminating qualities of the human visualsystem. It often happens that one or more light emitting elements ordisplay modules must be replaced due to damage, aging, or acts ofnature. A replacement light emitting element or display module often hasa different gray-scale, brightness and/or color response as the elementor module, respectively, that the light emitting element or displaymodule replaces. In-situ monitoring and calibration of a display isparticularly effective for maintaining uniformity in color, brightnessand grayscale across the entire viewing plane of the display, even whenreplacement of light emitting elements becomes necessary.

In general terms, in-situ display monitoring and calibration uses animage acquisition system to capture images of the viewing plane of thedisplay. Captured images may then be processed to characterize variousvisual performance characteristics of the display. When not in usecapturing images of the display, the image acquisition system can bestored in a manner that protects it from environmental hazards such asdust, dirt, precipitation, direct sunlight, etc. In addition, images maybe presented on the display that facilitate the calibration process. Forexample, a calibration image in which a plurality of light emittingelements is set to a particular color and intensity may be displayed, animage then captured, and then a difference between what was expected andwhat was captured may be developed for each light emitting element.Differences between captured images and expected images may be used tocreate a calibration data set which then may be used to adjust thedisplay of further images upon the display.

The visual performance of a display may be referenced from a definedviewpoint, which is essentially a point in 3 dimensional space fromwhich the viewing plane is viewed by a person. The image acquisitionsystem has an imaging plane for capturing images that is generally notat the same location as the viewpoint. A captured image may be postprocessed to infer what the display looks like at the viewpoint ofchoice. Each light emitting element of the display has a predeterminedposition and orientation in space. Each light emitting element producesan outgoing illuminance that varies in both brightness and colordepending on the distance to the viewer and on the angle between theviewer and the illuminance pattern produced by the light emittingelement. Knowing the distance, angles, and illuminance pattern between alight emitting element and an image plane enables the system to captureimages of the viewing plane on the imaging plane and then infer, bycomputations involving the know distance, angles, and illuminancepattern, what the viewing plane looks like when viewed from theviewpoint. Both monitoring of the display and calibration of the displayare thereby enabled by the system and methods of the present disclosure.

To further facilitate the present description it will be useful now toturn to the construction of a display according to various embodimentsof the present disclosure. Tessellation of a planar surface is thetiling of the plane using one or more geometric shapes, called tiles,creating no gaps and no overlaps. A periodic tiling has a repeatedgeometric pattern. A regular tiling is a tiling in which all tiles areregular polygons having the same size and shape. Square, triangular, andhexagonal tilings are each an example of a regular, periodic tiling thatcan achieve a tessellation of a planar surface without gaps or overlaps.Tilings are of special interest in the construction of modular displaysbecause their properties enable the construction of large displays withdesirable properties. Assembling a plurality of smaller display modulesin which each display module is configured to have a size, shape, andorientation corresponding to a predetermined tiling may produce a largedisplay having no gaps and no overlaps between adjacent display modules.

Within a single display module, a plurality of light emitting elementsmay be arranged in a predetermined pattern derived from an appropriatelyconfigured tiling. A planar tiling of regular polygons consists of edgesand vertexes. The set of vertexes of a regular polygon tiling can beseen to create a pattern with a high degree of regularity. A highlyuniform visual effect may be produced by placing a light emittingelement at or about each of the vertexes of a regular polygon tiling.

Light emitting elements of the present disclosure may each comprise asingle light emitting device or multiple light emitting devices. Apreferred light emitting element combines red, blue, and green lightemitting devices within one light emitting element so as to provide fullcolor spectrum display. Monochrome and other combinations of devices maybe used still within the spirit and scope of this disclosure. In otherembodiments a light emitting element may comprise white, red, blue andgreen devices within a single light emitting element. In otherembodiments a light emitting element may comprise red, green, blue andcyan devices. In other embodiments a light emitting element may comprisered, green, blue, yellow, and cyan devices, or any combination ofdevices emitting at different colors within a single light emittingelement. In other embodiments multiple devices emitting at substantiallythe same color may be used.

In still other embodiments of the present disclosure, light emittingelements may be replaced by light reflective elements. A lightreflective element may receive a portion of incoming ambient or directedlight and then reflect a portion of the light back to the viewer of adisplay. Modulating the reflective properties of the light reflectiveelement allows control over the intensity of the reflected light. Theportion of incoming ambient or directed light that is not reflected to aviewer may be absorbed, scattered or otherwise redirected so that it issubstantially attenuated with respect to a viewer of the display. Aplurality of light reflective elements may be modulated so as to produceimages upon a viewing plane. For a light source, a reflective displaysystem may use ambient light, directed non-ambient light, or acombination of both ambient and directed non-ambient light in producinga display.

In creating a uniform visual effect, it is useful to consider a propertycalled pitch distance, which is the distance between any light emittingelement and its closest adjacent light emitting elements. It can be seenthat a highly uniform visual effect is produced by maintaining a highlyuniform pitch throughout a single display module and across a pluralityof adjacent display modules. Preferred embodiments of the presentdisclosure use light emitting elements located at or about the vertexesof a regular polygon tiling. A regular square tiling is one suchpreferred tiling, producing a uniform visual effect by providing uniformspacing between both rows and columns of light emitting elements. Thespacing between adjacent rows and between adjacent columns of a regularsquare tiling may be referred to as the pitch of that pattern. In such asquare tiling, it can be seen that any light emitting element will haveat least two closest adjacent neighboring elements that are spaced apartfrom each other by a distance close to or substantially equal to thepitch distance.

In addition to uniform pitch within a single display module, the spacingbetween display modules can be controlled so that uniform pitch of lightemitting elements is maintained across a plurality of assembled displaymodules. A preferred embodiment is to provide a display module with aperimeter region, of a predetermined width, that contains no lightemitting elements. The preferred width of the perimeter region is lessthan or about equal to one half of the pitch distance, when measuredinward and along the edges of the regular polygon tiling defining thelocation of the plurality of the light emitting elements. When twodisplay modules are assembled adjacent to one another, each module mayprovide a perimeter region width of about one half of the pitch, whichcumulatively creates a pattern of uniform pitch spanning both modules. Aplurality of display modules may thereby be assembled to create uniformpitch spanning the plurality of display modules.

A single display module may comprise a plurality of light emittingelements coupled to a substrate, and arranged in a predetermined patterncorresponding to the vertexes of a regular polygon tiling. The displaymodule has a perimeter. A plurality of display modules may be assembledsuch that a portion of the perimeter of each display module abuts aportion of the perimeter of at least one other display module, eachmodule positioned to maintain uniform pitch spacing across the pluralityof display modules.

A display system according to the present disclosure may be constructedby assembling a plurality of display modules onto a support frame, thesupport frame having been previously constructed.

Turning now to FIG. 1A, shown is a regular four sided polygon, alsocalled a square 10, consistent with the square tiling 12 of the twodimensional plane shown in FIG. 1B. A coordinate system 8 is indicatedso as to make discussion of geometry features of the present disclosuremore clear. Square tiling 12 is comprised of a plurality of squaretiles, of which first square 10 a and second square 10 b are typical,arranged so that no gaps and no overlaps are produced. When arrangedinto the predetermined pattern shown in FIG. 1B, the square tiling 12can be seen to create a plurality of vertex 12 v and a plurality of side12 s, in which every vertex 12 v is separated a distance of about 12 sfrom each of its closest neighboring vertexes.

FIG. 1C shows predetermined pattern corresponding to a tiling of theplane 14 according to a square tiling. Overlaid onto the predeterminedpattern corresponding to a tiling of the plane 14 are x-axis 8X andy-axis 8Y, showing that a coordinate system can be overlaid onto thepredetermined pattern to facilitate clear disclosure of the location andalignment of other features to be described. The enlarged section,denoted FIG. 1D, shows that the square tiling of the plane gives rise toa highly uniform spacing of vertexes, which can be characterized aspitch distance 11. Pitch distance 11 corresponding to the predeterminedpattern 14 gives rise to uniform spacing between rows and columns whenthat predetermined pattern is based upon a square tiling. It can be seenthat row spacing and column spacing are both about equal to the pitchdistance 11.

Turning now to FIG. 1E, shown is a display module 70 having a pluralityof light emitting elements 72, of which first light emitting element 71a and second light emitting element 71 b are individual members of theplurality. Plurality of light emitting elements 72 is shown arrangedaccording to a predetermined pattern so as to create a highly uniformvisual effect upon display plane 74. FIG. 1F shows how predeterminedpattern 14 according to a square tiling of the plane may be used toposition individual light emitting elements 71 a, 71 b, and 71 caccording to the location of the vertexes of said predetermined pattern14. Superimposed upon the plurality of light emitting elements arex-axis 8X and y-axis 8Y. The display module 70 of FIG. 1F comprises aplurality of light emitting elements, each of which may be a singlelight emitting device or multiple light emitting devices. A preferredlight emitting element combines red, blue, and green light emittingdevices within one light emitting element so as to provide full colorspectrum display. Monochrome and other combinations of devices may beused still within the spirit and scope of this disclosure. The displaymodules of FIG. 1E and FIG. 1F each have a region adjacent to theirperimeter that is free from light emitting elements. This enables closespacing of adjacent modules as will be seen now.

FIG. 1G shows a first display module 70 a adjacent to a second displaymodule 70 b and disposed so that their display planes 74 a and 74 b abutand their respective y-axes 8Ya and 8Yb are substantially aligned,thereby creating a highly uniform visual effect that spans the combineddisplay modules. A pitch distance can be defined between adjacent lightemitting elements between adjacent display modules that is substantiallyequal to the pitch distance between adjacent light emitting elementswithin a single display module.

FIG. 1H shows a first display module 70 a adjacent to a second displaymodule 70 b and disposed so that their respective display planes 74 aand 74 b abut and their respective x-axes 8Xa and 8Xb are substantiallyaligned, thereby creating a highly uniform visual effect that spans thecombined display modules. A pitch distance can be defined betweenadjacent light emitting elements between adjacent display modules thatis substantially equal to the pitch distance between adjacent lightemitting elements within a single display module. When abutted andaligned in the foregoing manner, two adjacent modules may be combinedsuch that their combined plurality of light emitting elements aredisposed upon a single predetermined pattern 14 defining a regulartiling of the plane.

FIG. 1G and FIG. 1H make it clear that a large display may beconstructed from display modules designed according to the teaching ofFIG. 1A-FIG. 1H. Such a large display will tile the two dimensionalplane without gaps and without overlaps and produce a highly uniformvisual effect. Any number of display modules may be combined in both xand y directions to make a large display that is substantially free fromvisual aberrations.

Turning now to FIG. 2A, shown is a representative environment for usingin-situ display monitoring and calibration system 2. The figure shows aperspective view of a display 4, controlled by display control system 6,the display having a plurality of light emitting elements 72 disposed ina predetermined pattern collectively creating a viewing plane 80. Theplurality of light emitting elements may be formed in a predeterminedpattern according to any of the teachings of FIG. 1A-FIG. 1H. On thedisplay is shown representative visual media 28 rendered on viewingplane 80. Image acquisition system 30 is shown in a stored position 31.

FIG. 2B shows the display 4 of FIG. 2A and additionally shows imageacquisition system 30 in a deployed position 33 with a calibrationpattern 24 rendered on the viewing plane of display 4. Associated withthe display is display control system 6, which is operative to controlthe presentation of visual media on the display as well as to controlthe presentation of calibration patterns 24. The viewing plane 80 ofdisplay 4 in FIG. 2A and FIG. 2B has a predetermined geometric shape,geometric features G1, G2, G3, and G4 being associated with thatgeometric shape. In the embodiment of FIG. 2A and FIG. 2B, the geometricfeatures identified as G1, G2, G3, and G4 are corners of rectangularviewing plane 80. Other embodiments may have a viewing plane having adifferent shape and consequently may have other identifiable geometricfeatures that may be corners, edges, curved shapes or other identifiablegeometric features.

The deployed position 33 shown in FIG. 2B places image acquisitionsystem 30 in a predetermined position and orientation with respect tothe viewing plane. Said predetermined position and orientation issubstantially repeatable each time system 30 is moved to deployedposition 33. Causing image acquisition system 30 to cycle from deployedto stored to deployed position again results in disposing imageacquisition system in substantially the same position and orientation asin the previous deployed position.

Image acquisition system 30 is triggerable to capture one or more imageswhen the system is in the deployed position. When triggered, an imagemay be captured, the image comprising at least a portion of the viewingplane. In preferred embodiments the captured image comprises the entireviewing plane. In other preferred embodiments the image acquisitionsystem may comprise a plurality of imaging planes, each having a knowposition and orientation when in a deployed position, each operative tocapture an image of at least a portion of the viewing plane, theplurality of imaging planes operative to capture, collectively, theentire viewing plane.

Turning now to FIG. 3A, shown is a cross section view that was indicatedin FIG. 2B. In this view, image acquisition system 30 is shown indeployed position 33, imaging plane 36 being positioned and oriented tocapture one or more images of at least a portion of the viewing plane 80of display 4. In preferred embodiments, image acquisition system 30 maycomprise optical and electronic components known in the art for thecapture, storage, and transmission of high resolution digital imagessuitable for photometric applications. The image acquisition system ofFIG. 3A may comprise an actuatable linkage coupled between display 4 andimaging plane 36, the linkage being operable to move between the storedposition of FIG. 2A and deployed position 33 of FIG. 2B or FIG. 3A.Imaging plane 36 comprises an plurality of image receiving elementsarranged in a predetermined pattern. A jointed, powered, robotic arm isone of a number of feasible embodiments for actuatable linkage 20consistent with the operational requirements previously described.

The stored position of the image acquisition system may be furthercharacterized in that any electrical and optical components of the imageacquisition system contributing to or responsible for capturing imagesare substantially protected from exposure to environmental contaminantsincluding dust, dirt, moisture, direct sunlight, etc., that maydetrimentally affect the operation of the image acquisition system.

Continuing with FIG. 3A, shown are first incidence vector 50 a and firstviewpoint vector 86 a, both originating from a first light emittingelement 71 a comprising the viewing plane 80. Incidence vector 50 aoriginates at first light emitting element 71 a and points to the regionon imaging plane 36 that is operable to image first light emittingelement 71 a. In similar fashion, second incidence vector 50 b andsecond viewpoint vector 86 b originate at second light emitting element71 b, and third incidence vector 50 c and third viewpoint vector 86 coriginate at third light emitting element 71 c. Incidence vector 50 band 50 c originate at second and third light emitting elements 71 b and71 c, respectively, and each points to the region on imaging plane 36that is operable to image second and third light emitting elements 71 band 71 c, respectively. An incidence vector may be defined for eachlight emitting element comprising the viewing plane. It is evident inFIG. 3A that each incidence vector is influenced by the geometrical andoptical aspects of the position and orientation of imaging plane 36 withrespect to the position and orientation of viewing plane 80.

A viewpoint may be defined anywhere in three dimensional space fromwhich the viewing plane is visible. The viewpoint represents a viewerlocated at that distance looking at the viewing plane. For any given,fixed viewpoint, at each light emitting element a viewpoint vector maybe defined originating at the light emitting element and extending tothe viewpoint. For any given, fixed viewpoint, each light emittingelement may be expected to posses a unique viewpoint vector. It isevident from the geometry that a fixed viewpoint located far away fromthe viewing plane has the property that each viewpoint vector isessentially parallel to every other viewpoint vector. In FIG. 3A first,second, and third viewpoint vectors 86 a, 86 b, and 86 c, respectively,are drawn consistent with a viewpoint that is located far enough away sothat the viewpoint vectors are close to being parallel. In otherembodiments the viewpoint may be close enough to the viewing plane thatthe viewpoint vectors are not close to being parallel. It is evidentthat, regardless of how far the viewpoint is from the viewing plane, allviewpoint vectors converge at the viewpoint.

Each light emitting element produces a luminous flux that radiates awayfrom the light emitting element in 3 dimensional space. To facilitatethe discussion, a first surface normal vector may be defined thatoriginates at the location of the light emitting element and extendsperpendicular to the local curvature of the viewing plane. In addition,a second surface normal vector may be defined originating at a lightreceiving element comprising the imaging plane and extendingperpendicular to the imaging plane. The portion of a light emittingelement's luminous flux that is received remotely from the lightemitting element by a light receiving element having a given area isinversely proportional to the squared distance between emitter andreceiver, and is also a function not only of the brightness of the lightemitting element but also of the angle between the first surface normalvector and the second surface normal vector. It is evident that for anypredetermined position and orientation of the imaging plane, a uniqueincidence vector may be defined for each light emitting elementcomprising the viewing plane and that both angle and distance impact thelight that is received on the imaging plane by any particular lightemitting element.

An index i may be created for enumerating through each light emittingelement comprising the viewing plane. Index i may be allowed to take thevalues from 1 to N, where N is the total number of light emittingelements comprising the display. An incidence vector may therefor berepresented as: a_(i)*i+b_(i)*j+c_(i)*k; where {a_(i), b_(i), c_(i)} aredirection cosines corresponding to the i^(th) incidence vector, and {i,j, k} are unit vectors in x, y, and z directions, respectively.Furthermore, {x_(i), y_(i), z_(i)} describes location of the i^(th)light emitting element in 3 dimensions, and {x₀, y₀, z₀} describes thelocation of the center of the imaging plane in 3 dimensions. Thedistance from any particular light emitting element to the center of theimaging plane can be calculated as:D_(i)=[(x_(i)−x₀)²+(y_(i)−y₀)²+(z_(i)−z₀)²]^(1/2) Direction cosines{a_(i), b_(i), c_(i)} are accordingly determined by the formulas:a _(i)=(x _(i) −x ₀)/D _(i) ;b _(i)=(y _(i) −y ₀)/D _(i) ;c _(i)=(z _(i)−z ₀)/D _(i);

An even more exacting relationship can be described in which a uniquecoordinate {x_(0i), y_(0i), z_(0i)} on the imaging plane is associatedwith each light emitting element that is imaged. In that case thedistance be determined by the formula:D _(i)=[(x _(i) −x _(0i))²+(y _(i) −y _(0i))²+(z _(i) −z _(0i))²]^(1/2)

Direction cosines {a_(i), b_(i), c_(i)} are then determined bycomputing:a _(i)=(x _(i) −x _(0i))/D _(i) ;b _(i)=(y _(i) −y _(0i))/D _(i) ;c_(i)=(z _(i) −z _(0i))/D _(i);

FIG. 3B shows an enlarged view of the portion of a portion of imageacquisition system 30 in deployed position 33, as shown in of FIG. 3A.Visible in FIG. 3B is a plurality of image receiving elements 34 whichcollectively form imaging plane 36, and the dotted line paths ofincoming incidence vectors corresponding to first, second, and thirdincidence vectors previously described in connection with FIG. 3A. Inthe system of FIG. 3A and FIG. 3B, each of the plurality of lightemitting elements comprising the viewing plane 80 has associated with itits own incidence vector that points to the region of imaging plane thatimages the light received from each of the respective light emittingelements.

Shown now in FIG. 4A is a typical light emitting element 71. An X-X axisis indicated as 8X and a Y-Y axis is indicated as 8Y. Light emittingelement 71 produces a luminous output that diverges as it propagatesaway from the emitter at the speed of light. FIG. 4B is a schematicrepresentation of the effect on luminous flux that distance from theemitter makes. Luminous output 84 propagates away from light emittingelement 71. The same luminous output passing through area A0, located ata distance of r0 from the emitter, also passes through area A1, locateda distance r1 from area A0, and area A2, located a distance of r2 fromarea A0. Given luminous output 84 produced by light emitting element 71,the luminous flux measured by a light receiving element having a fixedsize will diminish with distance according to an inverse square law withrespect to distance from the emitter. A normalization operation can beperformed that compensates the measured value of luminous flux for thedistance dependence between the emitter and the receiver. Thus theluminous output received from light emitting elements that are atdifferent distances from the imaging plane can be directly comparedafter normalization.

FIG. 4C presents a graph of relative illuminance 35 versus radiationangle for the representative light emitting element 71 of FIG. 4A. Thereference designators X-X and Y-Y refer back to the light emittingelement of FIG. 4A. 0 degrees on the graph corresponds to a directionthat is perpendicular to the two dimensional plane containing both X-Xand Y-Y axes. On this graph the maximum illuminance has a value of 1.0,all other values being relative to this maximum. The graph indicates arepresentative way in which relative illuminance 35 will diminish as theangle with respect to either the X-X axis or the Y-Y axis moves awayfrom 0 degrees. A normalization operation can be performed thatcompensates for the reduction in relative illuminance caused by angle ofincidence between the incidence vector and relative illuminance. Thusthe luminous output received from light emitting elements that are atdifferent angles with respect to the imaging plane can be directlycompared after normalization. The angles of incidence with respect to x,y, and z axes can be determined using the direction cosines previouslydescribed.

It can be understood that the graph in FIG. 4C is an example of onespecific emitter for the purposes of teaching in this disclosure. Theexact pattern of radiation produced by any emitter is a multivariatefunction of the device or devices comprising emitter, the emitter'spackaging and how the emitter is mounted with respect to the viewingplane. Embodiments of an in-situ display monitoring and calibrationsystem according to the present disclosure may access configuration data29 corresponding to one or more of the following: radiation patternemitted by each emitter; position and orientation of each emitter; and,position and orientation of the imaging plane.

Turning now to FIG. 5A, shown is exemplary captured image 38, capturedby the image acquisition system. The image is of a rectangular viewingplane that has been captured with projection effects that are common forcamera and imaging systems. Visible in captured image 38 are geometricfeatures F1, F2, F3, and F4, which correspond to actual physicalfeatures present in the real world. In the embodiment of FIG. 5Afeatures F1, F2, F3, and F4 present in the captured image arecounterparts to real world features G1, G2, G3, and G4 visible in FIG.2B. Real world features G1, G2, G3, and G4 have associated with them {x,y, z} coordinates in three dimensions. The real world coordinates of G1,G2, G3, and G4 together with the image coordinates of F1, F2, F3, andF4, and calibration data from the imaging system allow an inverseprojection transformation to be computed. The inverse projectiontransformation of the captured image 38 in combination with incidencevector and distance data from each light emitting element, enable thecreation of a normalization function which can then be applied to acaptured image to produce, as shown in FIG. 5B, a normalized image 40.In essence, normalized image 40 is an estimate, based on the capturedimage, of what the display looks like from a single viewpoint which maybe distinct from position and orientation of the imaging plane. Thenormalized image may then be used as the basis for photometricoperations such as monitoring and/or calibration.

While the embodiments of FIG. 2-FIG. 5 have been described withreference to a single image acquisition system disposed in a positionwith respect to the display, other embodiments are within the scope ofthe disclosure. In other embodiments, the image acquisition system maybe moved through a sequence of different positions and/or orientations,capturing images from each different position and/or orientation in thesequence. Each different position or orientation of the imageacquisition system may be effective for capturing one or more images ofa different portion of the viewing plane. In some embodiments a sequenceof overlapping images may be acquired. Overlapping image portions may beused to establish registration between adjacent images. Both capturedimages and overlapping captured images may be used to establishregistration between a captured image and the portion of the viewingplane being imaged. A normalization function may be associated with eachposition and each orientation of the image acquisition system. Thus aplurality of normalized images may be acquired that collectively createsa normalized composite image of the entire viewing plane. Monitoring andcalibration may then proceed according to methods of the presentdisclosure.

FIG. 6 is a functional block diagram of an in-situ display monitoringand calibration system. The system 2 is shown comprising: an imageacquisition system 30 which is triggerable to capture one or morecaptured image 38 of the viewing plane of a display 4; a display controlsystem 6 having a data processing means 22 and a normalization function41, the display control system being operative to receive: visual mediadata 23; one or more calibration pattern 24; one or more expected image25 each of which is associated individually with at least one of saidone or more calibration patterns; one or more calibration data set 26; aconfiguration data set 29; one or more captured image 38; displaycontrol system 6 being further operative to produce transformed visualmedia data 27 that is rendered via display 4 on to the viewing plane ofsaid display as rendered visual media 28; display control system 6 beingfurther operative to: trigger the image acquisition system 30 to captureone or more captured image 38 of the viewing plane of display 4; producea normalized image 40 by transforming captured image 38 according tonormalization function 41; compare normalized image 40 to expected image25 and produce one or more calibration data sets 26; transform one ormore frames of visual media data 23 according to calibration data set 26and configuration data set 29 to produce transformed visual media data27. Visual media data 23 comprises brightness and color information foreach of a plurality of pixels at one or more encoded resolutions.Transformed visual media data 27 comprises brightness and colorinformation for each of a plurality of light emitting elements at adisplayed resolution.

The apparatus of FIG. 6 may include communications network 21, which maycomprise local and/or wide area networking components capable oftransmitting or receiving commands and/or data to local or remotedestinations. In conjunction with communications network 21, displaycontrol system 6 may be further operable to send, receive, and do dataprocessing operations on one or more of the following: visual mediadata, calibration pattern, expected image, calibration data set,configuration data, captured image, and normalized image.

FIG. 7 shows a flowchart for a process 200 of rendering visual media onthe viewing plane of a display that may be carried out by the in-situdisplay monitoring and calibration system. Process 200 comprises thesteps of:

202 receiving, in a display control system, a frame of visual media data23, the visual media data comprising brightness and color information atan encoded resolution for each picture element of a plurality of pictureelements arranged in a pattern corresponding to said encoded resolution;

204 transforming said frame of visual media data, in said displaycontrol system, using a calibration data set 26 to produce a frame oftransformed visual media data 27 at a displayed resolution for displayon a plurality of light emitting elements collectively forming a viewingplane of a display, said calibration data set comprising adjustments tobrightness and color for a plurality of light emitting elementscomprising said display;

206 displaying said transformed frame of visual media 28 on at least aportion of said viewing plane of said display.

FIG. 8 shows a flowchart for a process 300 of creating a normalizedimage that may be carried out by the in-situ display monitoring andcalibration system. Process 300 comprises the steps of:

302 positioning in a deployed position with respect to a display, animage acquisition system 30 having a plurality of light receivingelements collectively forming an imaging plane 36, the displaycomprising a plurality of light emitting elements arranged in apredetermined pattern collectively creating a viewing plane, such thatsaid viewing plane is imagable upon said imaging plane;

304 triggering the image acquisition system to acquire a captured image38 of said viewing plane;

306 defining an incidence vector 50 for each of said plurality of lightreceiving elements starting at each of said plurality of light emittingelements and directed toward the portion of the imaging plane thatimages each of said light emitting elements, each incidence vectorhaving both a direction and a magnitude;

308 associating with each of said light receiving elements comprisingthe imaging plane a normalization function 41 that compensatesbrightness and/or color differences in said direction and said magnitudefor each of said incidence vectors;

310 applying said normalization function 41 to the captured image 38thereby producing a normalized image 40.

FIG. 9 shows a flowchart of a process 400 of calibrating a display thatmay be carried out by the in-situ display monitoring and calibrationsystem. Process 400 comprises the steps of:

402 displaying a calibration pattern 24 on the viewing plane of adisplay;

404 associating an expected image 25 with said calibration pattern, saidexpected image comprising brightness and color information for eachlight emitting element comprising the viewing plane;

406 triggering an image acquisition system to acquire a captured image38 of said viewing plane;

408 creating a normalized image 40 from said captured image;

410 forming a calibration data set 26 comprising the color andbrightness differences between said expected image and said normalizedimage;

412 producing visual media rendered on the viewing plane of the display28 by applying, in a display control system, said calibration data set26 to the rendering of visual media 23 upon the viewing plane of saiddisplay such that the differences between said normalized image 40 andsaid expected image 25 are reduced.

Turning now to FIG. 10A, shown is a perspective view of an in-situdisplay monitoring and calibration system 2 with a calibration image 24displayed on viewing plane 80 of display 4. The system is showncomprising a first image acquisition system 30 a and a second imageacquisition system 30 b which collectively image the entire viewingplane. Both image acquisition systems are shown in deployed position 33.The display is shown having four geometric features G1, G2, G3, and G4.FIG. 10B shows a first captured image 38 a which may be captured byfirst image acquisition system 30 a of FIG. 10A. Geometric features F1and F4 are visible in first captured image 38 a and they correspond togeometric features G1 and G4, respectively, shown in FIG. 10A. FIG. 10Cshows a second captured image 38 b which may be captured by second imageacquisition system 30 b of FIG. 10A. Geometric features F2 and F3 arevisible in second captured image 38 b and they correspond to geometricfeatures G2 and G3, respectively, shown in FIG. 10A. Display controlsystem 6 is operable to: actuate both image acquisition systems betweendeployed 33 and stored positions; render one or more calibrationpatterns 24 on viewing plane 80, and trigger the capture of first andsecond image acquisition systems.

The display control system 6 of FIG. 10A may also be operable to:associate a first expected image received by first image acquisitionsystem 30 a with a portion of calibration pattern 24, associate a secondexpected image received by second image acquisition system 30 b with aportion of calibration pattern 24; produce the first normalized imagefrom first captured image; produce a second normalized image from thesecond capture image; produce a first calibration data set by comparingfirst normalized image to first expected image; produce a secondcalibration data set by comparing second normalized image to secondexpected image; combine first and second calibration data sets tocreating a composite calibration data set that comprises brightness andcolor corrections for each light emitting element comprising the viewingplane.

Embodiments like that disclosed in FIG. 10A may use a plurality of imageacquisition systems to capture images of different portions of a displayscreen, thereby collectively imaging the entire display. Otherembodiments may use a single image capture system that is moved todifferent positions in order to capture images of different portions ofthe display, the overall effect being to collectively image the entiredisplay while using just one image capture system.

Although the present invention has been described in considerable detailwith reference to certain preferred versions thereof, other versions arepossible. It may be desirable to combine features shown in variousembodiments into a single embodiment. A different number andconfiguration of features may be used to construct embodiments of theapparatus and systems that are entirely within the spirit and scope ofthe present disclosure. Therefor, the spirit and scope of the appendedclaims should not be limited to the description of the preferredversions contained herein.

Any element in a claim that does not explicitly state “means for”performing a specified function, or “step for” performing a specificfunction, is not to be interpreted as a “means” or “step” clause asspecified in 35 U.S.C. Section 112, Paragraph 6. In particular, the useof “step of” in the claims herein is not intended to invoke theprovisions of 35 U.S.C. Section 112, Paragraph 6.

What is claimed is:
 1. A system for use with a display, the displayhaving a plurality of light emitting elements arranged in apredetermined pattern collectively forming a viewing plane, the systemcomprising: a) a display control subsystem operative to display acalibration pattern on said viewing plane of said display, said displaycontrol subsystem associating an expected image with said calibrationpattern, said expected image comprising brightness and color informationfor each light emitting element comprising the viewing plane; b) a firstimage acquisition subsystem comprising a plurality of light receivingelements arranged in a predetermined pattern collectively forming afirst imaging plane; i) said first image acquisition system triggerableto capture a first captured image of a portion of said viewing plane;ii) said first imaging plane further characterized in that it is in aknown position and orientation with respect to said viewing plane whensaid first captured image is captured; c) a second image acquisitionsubsystem comprising a plurality of light receiving elements arranged ina predetermined pattern collectively forming a second imaging plane; i)said second image acquisition system triggerable to capture a secondcaptured image of a portion of said viewing plane; ii) said secondimaging plane further characterized in that it is in a known positionand orientation with respect to said viewing plane when said secondcaptured image is captured; d) the display control subsystem beingfurther configured to create, through data processing operations, both afirst normalized calibration image from said first captured image and asecond normalized calibration image from said second captured image,said display control subsystem further comprising: i) a first data setdefining a surface normal vector for each of said plurality of lightemitting elements, the surface normal vector originating at each lightemitting element and directed perpendicular to the viewing plane; ii) asecond data set defining an incidence vector for each of said pluralityof light emitting elements, a portion of said second data set definingan incidence vector originating at each of said plurality of lightemitting elements and directed toward said first imaging plane, anotherportion of said second data set defining an incidence vector originatingat each of said plurality of light emitting elements and directed towardthe said second imaging plane; iii) a third data set defining, for eachlight emitting element comprising said viewing plane: A) a firstluminous output directed in the direction of said surface normal vector;B) a second luminous output directed in the direction of said incidencevector; C) a known relationship that associates, quantitatively, saidsecond luminous output with said first luminous output; iv) said firstnormalized calibration image comprising an estimate of said firstluminous output for each of said plurality of light emitting elementsthat are captured by said first image acquisition system; v) said secondnormalized calibration image comprising an estimate of said firstluminous output for each of said plurality of light emitting elementsthat are captured by said second image acquisition system; e) saiddisplay control subsystem further operable to form a first calibrationdata set comprising at least one of {color or brightness} differencesbetween said expected image and said first normalized calibration image,the first calibration data set further comprising adjustments to atleast one of {color or brightness}, said adjustments corresponding tosaid differences between said expected image and said first normalizedcalibration image; f) said display control subsystem further operable toform a second calibration data set comprising at least one of {color orbrightness} differences between said expected image and said secondnormalized calibration image, the second calibration data set furthercomprising adjustments to at least one of {color or brightness}, saidadjustments corresponding to said differences between said expectedimage and said second normalized calibration image; g) said displaycontrol system further operable to apply said adjustments from saidfirst calibration data set and said second calibration data set to therendering of visual media thereby rendering transformed visual mediaupon said viewing plane of said display.
 2. The system of claim 1 inwhich both first luminous output and second luminous output arecharacterized in grayscale brightness.
 3. The system of claim 1 in whichboth first luminous output and second luminous output are characterizedin luminous intensity for each of a plurality of wavelengths of emittedlight.
 4. The system of claim 1 in which both first luminous output andsecond luminous output are characterized in wavelength distribution ofemitted light.
 5. The system of claim 1 additionally comprising: a) alinkage coupled to said first image acquisition subsystem, said linkagehaving a deployed position in which said first imaging plane is disposedat a known distance and orientation with respect to said viewing plane,said linkage also having a stored position in which said first imageacquisition subsystem is disposed to protect said first imaging planefrom airborne particulates and atmospheric moisture.
 6. The system ofclaim 1 in which the system is further operable to wait for apre-established time-of-day before displaying said calibration patternon said viewing plane.
 7. The system of claim 1 in which the system isfurther operable to wait for the absence of weather conditions includingprecipitation, in proximity to said display, before displaying saidcalibration pattern on said viewing plane.
 8. The system of claim 1further characterized in that said display control subsystem associatessaid calibration pattern with a plurality of expected images, eachexpected image of said plurality of expected images corresponding to adifferent operating temperature of the display, the system being furtheroperative to select one of the plurality of expected images according tothe present operating temperature of the display, the selected expectedimage comprising brightness and color information for each lightemitting element comprising the viewing plane.
 9. The system of claim 1further characterized in that: a) the first image acquisition subsystemis further operative to capture a plurality of captured images; b) thedisplay control subsystem is further operative to: create a statisticalcombination of said plurality of captured images; and, create the firstnormalized calibration image from said statistical combination of saidplurality of captured images.
 10. A system for use with a display, thedisplay having a plurality of display modules, each display modulehaving a plurality of light emitting elements arranged in apredetermined pattern collectively forming a viewing plane, the systemcomprising: a) a display control subsystem operative to display acalibration pattern on said viewing plane of said display, said displaycontrol subsystem associating an expected image with said calibrationpattern, said expected image comprising brightness and color informationfor each light emitting element comprising the viewing plane; b) a firstimage acquisition subsystem comprising a plurality of light receivingelements arranged in a predetermined pattern collectively forming afirst imaging plane; i) said first image acquisition system triggerableto capture a first captured image of a portion of said viewing plane;ii) said first imaging plane further characterized in that it is in aknown position and orientation with respect to said viewing plane whensaid first captured image is captured; c) a second image acquisitionsubsystem comprising a plurality of light receiving elements arranged ina predetermined pattern collectively forming a second imaging plane; i)said second image acquisition system triggerable to capture a secondcaptured image of a portion of said viewing plane; ii) said secondimaging plane further characterized in that it is in a known positionand orientation with respect to said viewing plane when said secondcaptured image is captured; d) the display control subsystem beingfurther configured to create, through data processing operations, both afirst normalized calibration image from said first captured image and asecond normalized calibration image from said second captured image,said display control subsystem further comprising: i) a first data setdefining a surface normal vector for each of said plurality of displaymodules, the surface normal vector originating at about the centroid ofeach of said plurality of display modules and directed perpendicular tothe viewing plane; ii) a second data set defining an incidence vectorfor each of said plurality of light emitting elements, a portion of saidsecond data set defining an incidence vector originating at about thecentroid of each of said plurality of display modules and directedtoward said first imaging plane, another portion of said second data setdefining an incidence vector originating at about the centroid of eachof said plurality of display modules and directed toward the said secondimaging plane; iii) a third data set defining, for each light emittingelement comprising said viewing plane: A) a first luminous outputdirected in the direction of said surface normal vector; B) a secondluminous output directed in the direction of said incidence vector; C) aknown relationship that associates, quantitatively, said second luminousoutput with said first luminous output; iv) said first normalizedcalibration image comprising an estimate of said first luminous outputfor each of said plurality of light emitting elements that are capturedby said first image acquisition system; v) said second normalizedcalibration image comprising an estimate of said first luminous outputfor each of said plurality of light emitting elements that are capturedby said second image acquisition system; e) said display controlsubsystem further operable to form a first calibration data setcomprising at least one of {color or brightness} differences betweensaid expected image and said first normalized calibration image, thefirst calibration data set further comprising adjustments to at leastone of {color or brightness}, said adjustments corresponding to saiddifferences between said expected image and said first normalizedcalibration image; f) said display control subsystem further operable toform a second calibration data set comprising at least one of {color orbrightness} differences between said expected image and said secondnormalized calibration image, the second calibration data set furthercomprising adjustments to at least one of {color or brightness}, saidadjustments corresponding to said differences between said expectedimage and said second normalized calibration image; g) said displaycontrol system further operable to apply said adjustments from saidfirst calibration data set and said second calibration data set to therendering of visual media thereby rendering transformed visual mediaupon said viewing plane of said display.
 11. The system of claim 10 inwhich both first luminous output and second luminous output arecharacterized in grayscale brightness.
 12. The system of claim 10 inwhich both first luminous output and second luminous output arecharacterized in luminous intensity for each of a plurality ofwavelengths of emitted light.
 13. The system of claim 10 in which bothfirst luminous output and second luminous output are characterized inwavelength distribution of emitted light.
 14. The system of claim 10additionally comprising: a) a linkage coupled to said first imageacquisition subsystem, said linkage having a deployed position in whichsaid first imaging plane is disposed at a known distance and orientationwith respect to said viewing plane, said linkage also having a storedposition in which said first image acquisition subsystem is disposed toprotect said first imaging plane from airborne particulates andatmospheric moisture.
 15. The system of claim 10 in which the system isfurther operable to wait for a pre-established time-of-day beforedisplaying said calibration pattern on said viewing plane.
 16. Thesystem of claim 10 in which the system is further operable to wait forthe absence of weather conditions including precipitation, in proximityto said display, before displaying said calibration pattern on saidviewing plane.
 17. The system of claim 10 further characterized in thatsaid display control subsystem associates said calibration pattern witha plurality of expected images, each expected image of said plurality ofexpected images corresponding to a different operating temperature ofthe display, the system being further operative to select one of theplurality of expected images according to the present operatingtemperature of the display, the selected expected image comprisingbrightness and color information for each light emitting elementcomprising the viewing plane.
 18. The system of claim 10 furthercharacterized in that: a) the first image acquisition subsystem isfurther operative to capture a plurality of captured images; b) thedisplay control subsystem is further operative to: create a statisticalcombination of said plurality of captured images; and, create the firstnormalized calibration image from said statistical combination of saidplurality of captured images.